In many cases, it is very important for the system being used to identify its own position and orientation in space. For a solid object, position and orientation have for example the following 6 Degrees Of Freedom (“DOF”): translation (x, y, z) and rotation (yaw, pitch, roll). These six DOF may be defined relatively to a first position measured in time (or relatively to earth by using GPS).
There are various methods that are designed to enable a system finding its own 6 DOF in time, while moving the system in order to find its 6 DOF. One such method is by using a camera for capturing sequential frames and then analyze the differences between the frames, in order to retrieve the camera shift that occurred between the frames taken and aggregate those shifts in order to determine a current position relative to the initial position.
Another option is to use an Inertial Measurement Unit (“IMU”) which includes accelerometers, gyros and magnetic compass, in order to measure the system's movement. Moreover, it is possible to use a combination that comprises one or a few cameras and one or a few IMUs, in order to improve the accuracy of the position detection.
Various applications require the use of such systems. For example, Augmented Reality (AR) and Virtual Reality (VR), in which a system is located on top of the user's head for measuring the movements of the user's head, where movements are defined as translation and/or rotation.
In the case of Augmented Reality, knowledge about the exact head position enables rendering an artificial object within the world around the user. In this scenario, one can mount a device on one's head, e.g. in the form of AR glasses, and the device would display an object onto the AR glasses. In order to simulate a real object in the world that remains at the same position, being independent of different head positions, the device requires the knowledge of the head position. If, for example, the head is turned to left, the augmented object should be moved to a right position from its previous location at the AR glasses, whereas if the head is turned to the right, the augmented object should be moved to a left position from its previous location at the AR glasses.
For the case of Virtual Reality, in order to avoid nausea and sickness, one should measure the head movement and render a virtual world relative to the head position, in a way that the world changes essentially according to the user's head movements.
Another example for such an application can be a 3D reconstruction, in which the user can scan an object from different angles and create a 3D model of the object from these scanning results. One can use a camera of a mobile phone or a drone in order to scan small or big objects. Knowing the exact camera position in such applications enables retrieving data from the different images obtained and aggregate it into one exact 3D model.
Under poor light conditions, the above-described systems can operate within the Infra-Red (IR) spectrum, by illuminating the environment with an IR projector that does not interfere with the human eye, yet enable an IR camera to view the environment, and the system to determine its position by analyzing the shift occurring between the frames captured by the IR camera. Using depth cameras is also common in such cases, because depth cameras may provide the exact 3D position of the respective objects in the world.
All of the above applications can be in the form of embedded solutions, where the power supply is a battery. In these cases of operating under a dark environment, where the IR projector might consume most of the battery power, reducing its power consumption is a factor that has a critical effect on the systems' ability to function for a long period of time.
The typical prior art solutions for this matter are either using RGB cameras having no projectors which do not function satisfactorily under poor light conditions, or using a projector while capturing all images, resulting in a high-power consumption. Still, in order to reduce power consumption in these systems, one may reduce the frame rate of the images being captured in the attempt to rely extensively on extracting the camera's position from IMU readings. Reducing the frame rate is required to allow increasing the integration time (per frame), in order to get reasonably decent signals under poor light conditions. This in turn leads to smeared images, and consequently reduction of SLAM precision and increase SLAM sensing latency.
The term “SLAM” as used herein stands for Simultaneous Localization and Mapping. Localization relates to finding the current camera position. Mapping refers to saving the positions of objects in a scene so that when one returns to the same scene, he/she would not have to scan the whole scene again and instead will be able to retrieve objects' location from a database (a map of the scene that was saved).
The present invention seeks to provide a solution that will reduce power consumption when operating under poor light conditions while overcoming the problems described above, thereby enabling such systems to operate for longer periods of time.